In video processing for stereo or 3D reproduction, one issue is the color difference between two or more views of the same scene as 3D video content is often created from two or more captured 2D videos. These differences may result for example from physical light effects or from cameras being not complete identical as e.g. each camera has its own lens, sensors and further specific behavior. Especially in case of 3D reproduction, it leads to disturbing effects that both pictures have a slightly different color which may cause pain in the head of the viewers.
Furthermore, there are several technical aspects for which calibrated colors of stereo images are desired as a compensation of color differences e.g. reduces the required bitrate, allows a more precise disparity estimation to create or enhance 3D information or 2D images using 3D information for view interpolation or a detection of hidden objects.
Known methods for compensating color differences between input images can be divided into two groups: color mapping and color transfer. Usually, two images are processed and the goal is to describe the color transformation that allows transforming colors of one image into the colors of the other image of the same scene.
In color mapping, it is assumed that geometrically correspondences—so-called feature correspondences between the input images are available. A well-known method for feature correspondences is Scale Invariant Feature Transform, so-called SIFT. It detects corresponding feature points using a descriptor based on a Difference of Gaussian, so-called DoG, in the input images. Geometrical correspondences are often not available in images or parts of images that are low textured, for example sky, surfaces of man-made, uni-colored images.
In color transfer, geometrical correspondences are not used and images are not required to be textured. There is a case where precise geometrical correspondences are not meaningful because the two input images do not show the same semantic scene but are just semantically close. According to a well-known color transfer algorithm, first and second order image signal statics are transferred from a reference image to the corresponding target image. In order to be able to process color channels separately, an empirical, de-correlated color space is used.
That means, when applying a known color mapping algorithm, if the image content in a part of the image does not correspond to the selection criteria of the SIFT algorithm, no colors from this part of the image will be exploited. This is the case, for example, in low textured parts of an image.
And, when applying a color transfer method to images that show the same semantic scene, the precision of the calculated color transform will suffer from the presence of image regions that have no correspondence in the other image, respectively, as e.g. image statistics will be influenced by such regions. This is the case or example for stereo images where parts at the left border of the left image may not be visible in the right image and vice versa. Another example is images from a motion picture scene where the camera motion is travelling type. Here, in each image, a small part of the scene is not any longer visible and another small part of the scene becomes visible but has not been before.